Method and apparatus for conversion of sound signals into light

ABSTRACT

Method and apparatus for the conversion of sound waves to electromagnetic wave forms, preferably light, whereby sound waves are converted to an electrical signal and processed by a number of filters, the distribution between the filters being a result of the frequency of the sound wave and in which the filters are subsequently connected to their respective color display and where the individual color display&#39;s activation is directly proportional to their filter&#39;s amount of signal processing and where the color display visualization in a display means is in the form of a single color or a mixture of two or more color displays.

The invention concerns a method for the conversion of soundwaves intoelectromagnetic wave movement, preferably light, whereby soundwaves areconverted to an electrical signal and are processed by a number offilters, and in addition an apparatus for performing the method.

U.S. Pat. No. 5,191,319 discloses a system for filtering music in 11variable width frequency bands, in which every interval results in apreset colour display. In this patent the colours are chosen from whatvisually looks best.

U.S. Pat. No. 4,614,942 discloses a system as the above, but where afourband model is used, in which one similarly chooses a colourvisualisation based on sound influences based on what seems mostvisually appropriate. In other words, the state of the art showsconverting of sound into light, but where a signal may only imply thatone colour is activated, and thereby does not give the possibility forblending of colours, where the colour mixture will assume differentappearances, depending on from which frequency the sound originates.

It is the purpose of the invention to create a method which does nothave the disadvantages of the previous mentioned systems, and where itis possible to convert sound to light by means of colours in such a waythat a specified frequency will represent a single colour or mixtures ofcolours.

This purpose is achieved by the method referred to in the introductionand where, in addition, one filter processes a part of the electricalsignal, the size of which part being dependent of the frequency of thesound wave, and also that the remaining filters process the remainingpart of the electrical signal, whose distribution among the remainingfilters is similarly a function of the frequency of the sound wave,which filters subsequently each are connected to respective colourdisplays, where the colour display of each filter is a predeterminedcolour, and where the generating of each single colour display isdirectly proportional to the part of the signal processesing of thecorresponding filter, which colour displays are visualized in a colourdisplay means as a single colour or as a mixture of two or more colourdisplays.

By such a method it is possible to convert any sound to a light image,possibly on a computer screen or a light emission source. Depending onthe tone, a sound will show itself as an image of an individual colouror combination of colours, in that a sound tone will result in a singleor more filters being activated and where each filter is connected tocolour displays. For example, if a filter is connected to a colourdisplay that is blue, together with another filter colour display whichis yellow and if the filters are activated in the ratio 1:1, the outputon the display means will be green. It is thereby possible to achieve ainfinitely variable, visual registration of a sound signal which can beused with sound shows, that are to be visualized, and in connection withdeaf-handicapped who, by this process, can achieve an understanding andawareness of sound.

By using such a method according to the invention; it allows thepossibility of mixing only two colours together with the individualchosen colour, which results in an unambiguous signal andreading/decoding of the sound. This is important especially inconnection with the use of the invention by deaf-handicapped.

By using a method according to the invention an appropriate processingof sound which is easy to visualize is achieved.

By using a method according to the invention an appropriate means ofvisualizing the whole audible sound spectrum is achieved.

All colours may be expressed either as a frequency (Hertz) or aswavelengths (nanometers). This applies similarly to the audible soundareas, where wavelengths are expressed i m, cm, or mm. The human ear candetect sound from approx 20 Hz to approx 20 kHz (wave-lengths from 20 mto 20 mm) corresponding to 10 octaves. The human eye can registerwavelengths from approx 792 nm to 396 nm, corresponding to one doublingof the frequency or 1 audible octave. The optimal display is thereforedevided in 10 sections.

In order to visualize the complete audible sound spectrum there is needfor 10 displays representing 10 succeeding frequency doublings. Eachdisplay represents the whole colour spectrum through the three primarycolours red, green, and blue.

If we spectrally convert all the audible frequencies simultaneously(white noise) the result will be white light (equal parts of red, green,blue). Silence will be equivalent to darkness.

By the use of the method according to the invention the use of aspecific and up to date displaying apparatus will be achieved.

By the use of the method as disclosed, it is possible to visuallyrealize not only the individual tones, but also to show, in whichinterval the tone takes place, in other words whether the octave usedbelongs to low or high frequencies. This is especially useful for thedeaf.

In order to achieve an optimal visualization of sound the colour displayapparatus is constructed in such a way so that low frequencies aredisplayed at the bottom and high frequencies at the top. This makes itpossible to see more frequencies at the same time, thus it is possible,for example, to see overtone spectra of individual sounds or severaldifferent sounds, voices and/or instruments at the same time.

The invention in addition relates to an apparatus for the effectuationof this process.

The human voice has a complex oscillisation structure, containingfundamental tones, overtones, vowels, consonants and formants, whichwill all be visible in several of the converters display and octaveareas simultaneously.

Differences in human physique and psychology results in human voicessounding differently. This also means that different voices also arevisualized differently even though they are singing the same note intothe apparatus.

By means of the colour display apparatus the spectator can learn to seeand also remember a specific colour combination expressing specific toneshades. In this way an auditive impression can be experienced togetherwith visual impressions.

The invention will now be described in more detail with references tothe drawings, where

FIG. 1 the principal layout plan for the conversion,

FIG. 2 illustrates a filter configuration for a prototype where a filterwith a step characteristic has been chosen,

FIG. 3 illustrates the limiter function for the conversion,

FIG. 4 the signal path of the converter,

FIG. 5 illustrates the state-variable filter,

FIG. 6 illustrates the input board,

FIG. 7 illustrates the basic filter board,

FIG. 8 illustrates the in/out board,

FIG. 9 illustrates the single compressor/limiter,

FIG. 10 illustrates the conversion diagram and filter configuration foranother prototype.

FIG. 1 illustrates the conversion possibilities for a sound, in which wehave a filtercard able to analyse electrically presented sound sources.A light control component consisting of light dimmers with relevantlight sources. Each filtercard is connected to 3 light dimmers withtheir respective 3 lightsources in the 3 primary colours. A displaycomponent consisting of transparent material (plastic, plexiglas, glassetc.) or alternatively a white surface upon which the three primarycolours can be mixed or projected.

The conversion analysis component contains a number of filter cards.Each filter card comprises 3 filters and each card has 3 outputs--onefor each primary colour. Outputs are suitable for controlling standardlight dimmers with control voltage 0-10 VDC. When the filter card isactivated by an electrically presented sound signal, the filter analysesthe frequency and dynamics. By way of the card's conversion factor thiselectrical current is distributed to the three outputs of the filtercards. In this way the 3 light dimmers are activated by the controlvoltages conditioned on frequency and dynamics. The 3 light sourcesconnected to respective filter cards reproduce these frequencies anddynamics as visible light.

The prototype of the converter is fitted with 6 filtercards, all workingidentically. Each individual filtercard is set to process 6 oscillationsdoublings in succession between 130,8 Hz and 8371,2 Hz. Each filtercardis connected to 3 light dimmers which produces in total 3×6 lightdisplays=18. These 6 light displays are focused on each of theirrespective displays. Each set of light displays consists of 3 primarycolours red, green and blue, with wavelengths respectively of 720 nm(red), 539 nm (green) and 453 nm (blue). Through an analysis of thewhole sound spectrum within a frequency doubling (for example 440 Hz to880 Hz) it is possible for the light sources, through variable controlvoltages, to represent the complete visible colour spectrum.

On the basis of this oscillation doubling principle in the converter,any sound frequency will be represented unambiguously in the colourspectrum by means of the converter. In this system it has been possibleto make a complete linear conversion between sound and light and becauseall filters are constructed analoguosly all light shade transisitionsare completely even (gradual).

The conversion principle of the converter is founded on the recognitionof the natural structure of sound and light and the subsequentconnection. This connection means that every frequency will represent aspecific colour, and that any sound, including over and under tonespectra, reverbation and acoustic circumstances will also represent aspecific colour.

The process together with the apparatus is fully analogue constructed toinsure the fastest reaction response to the conversion. Each filter isconstructed using state-variable filter technology, which gives theoptimal phase response to audio. Unlike tripotientmeters, which quicklylose aligment, there is presently used measured, hand built resistorswhich display great reliability.

EXAMPLE a.

At the outset: A wavelength of 75 cm corresponding to a pure sinus tonewith the frequency 440 Hz. The wavelength is halved to 37,5 cm resultingin that the frequency is doubled to 880 Hz. By carrying out wavelengthdivision 20 times, the wavelength is reduced to 719 nm corresponding toan oscillation of 461.373 kHz. After 5-6 oscillation doubling, approx20.000 kHz, the frequency is no longer audible to the human ear. Theoscillation frequency of 264.373 kHz is visible to the human eye as redlight.

The conversion factor from sound to light is dependent on, whichfrequency/frequencies is used as input, dependent on where the number offrequency doublings, the sound or sounds to be converted, is positionedfrom the visible spectrum frequency. In this way through the calculationfactor. a direct transformation function between sound and light iscreated.

EXAMPLE b.

As in example a the starting point is a pure sinus tone of 440 Hz, whichin musical terminology is equivalent to the note "A". We now halve thewavelength to 37,5 cm=880 Hz. The tone is still A, only an octavehigher. If we carry on halving wavelengths 20 times, the final wavewould be 719 nm=461.373 kHz equivalent to red light. As already stated,after 5-6 octaves the frequency is out of the audible range of the humanear. We are unable to hear the tone, but we still allow calling it A aswe repeat octaves. After having octave doubled 20 times from 440 Hz wearrive at a point where "A" is visible corresponding to the colour redat 719 nm=461.737 kHz. Each frequency/tone has a specific colour.

As the starting point is a pure sinus tone the converted result will bedefinitively red. The majority of sounds we know, have a more complexwave structure. Therefore the result of the same note "A"/440 Hz stillwill be red (basic tone), but in addition a variety of overtone spectrawill be present, resulting in a number of colours will represent thehigher frequencies (overtones) together with the lower frequencies(undertones) depending on which sound source is used.

FIG. 2 illustrates how band-pass filters are arranged in connection toeach other. The filters referenced to have a slope, in which the signalis 24 dB below each filter top. This is necessary due to the insulationdemands between two individual tones, which are to be registered.

The filters are constructed after the state-variable principle which isillustrated in FIG. 5, whereby it is possible to achieve the necessaryfilter slope and appropriate phase relationships in the transitionfrequencies.

FIG. 7 illustrates a print board drawing of the filter as set out inFIG. 2 and FIG. 5.

The system is designed to process a complete octave, in other words 12halfnotes for each filtercard. It is therefore nessecary, that thecenter frequency of each note is placed exactly at the resonance top ofthe related filter and immediately after falls sharply before the nextfilter. The filters must not have a smaller Q factor than that foravoiding oscillation in the filters. This results in an compromiseevaluation in relation to the slope/ringing of the filters and isdifferent depending on which note is involved. All the filters aretherefore precisely adjusted with handfiled 1% metal film resistors,both for accuracy in the filterfrequency and the band width, which isreferred to as Q.

After that all the filter cards have been adjusted to their respectivefilterfrequencies, the corresponding mixing levels are (red, green,blue) are adjusted.

Mixing of colours is achieved by aggregating the pure amplitudemodulated signals from the 12 filters in 3 different virtual earthsumming amps, respectively called red--green and blue sum amps. Fromeach filter a total of 3 resistors are connected to a semi balancedsumming bus, respectively, and depending on how the 3 resistors relativeOhm values are set, these will enter the 3 sum amps at a precisely setlevel.

The 3 sum amps are followed by an A/C convertor, which converts radiosignals to DC current from 0-10 VDC. This scale has been chosen becauseit matches to nearly all existing lighting equipment.

This DC current is sent from the apparatus to an ordinary light systemcontaining triac-controls for incandescent lamps.

If in the first instance, only one frequency doubling is used, 3 lampswill be used, namely red, green and blue. These 3 triacs receive theircurrent from the 3 sum amps.

When the filter card receive a tone (note), for example an A, the filterA will allow the tone to pass, while the other filters will block thisfrequency. In accordance with the table above the tone A equals red.

Though it was previously mentioned that each filter had 3 resistorsconnected to the sum bus, in the case of tone A it is only necessarywith 1 resistor to the red sum amps.

The signal from the red sum amps will be subsequently be rectified andsent as DC current to the triac, which makes the red lamp to light up.

If alternatively the note of C is sent to the soundcard, the filter Cwill likewise allow the tone to pass and the other filters will in turnblock for this particular tone. The note (tone) C represents the colouryellow, which is a mixture of 50% red and 50% green. In this case thefilter signal passes down to the red and green summing bus through 2resistors, whose mutual related values are 50% and 50%. As previouslyreferred to, the signals end up as DC current, and now both red andgreen lamps are illuminated, which, when mixed on a white surface orprojected through a transparent medium will produce the colour yellow.

FIG. 6 and FIG. 4 shows an input board. FIG. 8 illustrates the printboard for the input board. This board consists of a stereo line inputand a mono microphone input. These inputs are all electronicallybalanced in order to avoid outside interference noise and other possiblesignal problems, when using long cable lengths to and from theapparatus.

The possibility is also present for mixing line and microphone signalstogether when both switches are activated at the same time.

The actual principal of the input board is that the line input isreceived in stereo and relayed to the built in stereo mixer, to whichthe mono microphone signal arrives. This microphone signal is sent toboth left and right channels, so that it always appears in the middle ofthe stereo signal. From there the signal is relayed to a stereo outputstep, where the line and microphone signals emerge as mixed. This stereooutput ends in 2 jack sticks at the back of the apparatus and are usedto connect a stereo amplifier with its related speakers. It is notpossible to change the level of the line signal, since it ispreconditioned that as the input level is placed between -10 to 0 dB.

The microphone input has however a gain-potentiometer at the front. Thishas a scale from -50 to +10 dB. At this input an 18 volt phantom-voltageis operative, when using a microphone of the condensor type. Thephantom-voltage cannot be turned off, but has no consequence for theoperation of dynamic microphones and cannot damage them in any way.

From the mixer component the stereo signal is divided into 2 lines. Astereo signal is relayed to the previously mentioned output step, and amono-mix of left and right is sent from the input board to the limiterboard, where one has the possibility of adjusting compression drive andoutput level. From here these are returned to the input board, where themono signal is distributed and subdivided to 6 seperate amplifiers, withindividual related trimmer controls at the front. Each of theseamplifiers exits from the board to their related filter boards, which inthe mentioned system are 6 in number.

Limiter Function for the Converter.

FIG. 3 illustrates how the limiter board processes the incoming audio.FIG. 9 illustrates the print board. As it appears from the figure, weare not talking about a real limiter, but about a compression of theaudio signals with such a large ratio as it becomes an approximation ofa limiter curve. This is necessary in order to adapt dynamic audiblesound to the often rather less dynamic light spectrum.

Alternative Embodiments of the Converter.

In addition to the above other embodiments can also be used. For examplea version which includes 3 filters per oscillation doubling instead ofthe 12 which the prototype is equipped with. The bandwidth and slope ofthe three filters refer directly to the frequency-related position inthe light spectrum of the 3 primary colours, see also FIG. 10. Thisshows the position of the primary colours (red, green and blue)placement in relation to one frequency doubling together with thefilter's slope for this prototype using 3 filters per oscillationdoubling and directly referring to the visible light spectrum.

In addition a PC-based digital version is very well suited. This modelis based on the same basic principles as the analogue model, but canbetter meet specific demands from users and have a great degree offlexibility in relation to the areas of analysis (frequencies--eventhose out of the audible spectrum), the possibility of colour-freezing,repetition of frequency changes and colour combinations etc.

A PC version will open up the possibilities of running the convertertogether with already existing analyzing tools as used in connectionwith speech teaching.

Future versions will also be able to use an ordinary TV for example, awide screen projector or a monitor as a display/mixing medium.

For people whose hearing abilities are partially or completely impaired,the conversion is a new method for training language and auditiveorientation, amongst other things as an articulation tool. In connectionwith work amongst the physically and psychologically handicapped theconversion also acts as a concentration and motivation tool.

For the hearing human the conversion is a means for more intenseawareness of music, music understanding and a new visual sound dimensionin daily life for relaxation, entertainment, immersion or enjoyment.

I claim:
 1. In a method for converting sound waves intoelectromagnetical wave movements comprising light, which sound waves areconverted to an electrical signal and are processed in a number offilters, where one filter processes a part of the electrical signal, thesize of the part being dependent of the frequency of the sound wave, andwhere the remaining filters process remaining parts of the electricalsignal, distribution among the remaining filters being similarly afunction of the frequency of the sound wave, the filters subsequentlyeach being connected to respective colour displays, where the colourdisplay of each filter is a predetermined colour, and where thegenerating of each single colour display is directly proportional to thepart of the signal processed by the corresponding filter, the colourdisplays being visualized in a colour display means as a single colouror as a mixture of two or more colour displays, where the electricalsignal is divided into intervals each being processed by three filters,the improvement comprising that each interval spans a frequency doublingof the original sound wave.
 2. A method according to claim 1, whereinfor each sound interval the colour displays are generated in differentand predetermined sections on the colour display means.
 3. A methodaccording to claim 1, wherein position, bandwidth, and slope of thethree filters in combination spanning a single interval corresponddirectly to the frequency-related position in the light spectrum of thethree primary colours.